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Mission 

Our  mission  is  to  provide  independent,  relevant,  and  timely  oversight 
of  the  Department  of  Defense  that  supports  the  warfighter;  promotes 
accountability,  integrity,  and  efficiency;  advises  the  Secretary  of 
Defense  and  Congress;  and  informs  the  public. 


Vision 

Our  vision  is  to  be  a  model  oversight  organization  in  the  Federal 
Government  by  leading  change,  speaking  truth,  and  promoting 
excellence — a  diverse  organization,  working  together  as  one 
professional  team,  recognized  as  leaders  in  our  field. 


Fraud,  Waste  &  Abuse 

HOTLINE 

Department  of  Defense 

dodig.mil/hotlinel8oo.424.9098 


For  more  information  about  whistleblower  protection,  please  see  the  inside  back  cover. 


INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 

4800  MARK  CENTER  DRIVE 
ALEXANDRIA,  VIRGINIA  22350-1500 


December  9,  2014 

MEMORANDUM  FOR  AUDITOR  GENERAL,  DEPARTMENT  OF  THE  AIR  FORCE 

SUBJECT:  Quality  Control  Review  of  Air  Force  Audit  Agency's  Special  Access 
Program  Audits  (Report  No.  DODIG-2015-049) 

We  are  providing  this  report  for  your  information  and  use.  The  generally  accepted  government 
auditing  standards  (GAGAS)  require  that  an  audit  organization  performing  audits  and/or 
attestation  in  accordance  with  GAGAS  should  have  an  appropriate  internal  quality  control 
system  in  place  and  undergo  an  external  peer  review  at  least  once  every  three  years  by 
reviewers  independent  of  the  audit  organization  being  reviewed.  As  the  organization  that  has 
audit  policy  and  oversight  responsibilities  for  audits  in  the  DoD,  we  conducted  the  external 
quality  control  review  of  the  AFAA  SAP  audits  in  conjunction  with  the  Army  Audit  Agency's 
review  of  the  AFAA  non-SAP  audits.  We  conducted  this  quality  control  review  in  accordance 
with  the  Council  of  the  Inspectors  General  on  Integrity  and  Efficiency  (CIGIE)  Quality 
Standards  for  Inspection  and  Evaluation. 

An  audit  organization's  quality  control  policies  and  procedures  should  be  appropriately 
comprehensive  and  suitably  designed  to  provide  reasonable  assurance  of  meeting  the 
objectives  of  quality  control.  We  tested  the  AFAA  SAP  system  of  quality  control  for  audits 
to  the  extent  considered  appropriate. 

Federal  audit  organizations  can  receive  a  rating  of  pass ,  pass  with  deficiencies ,  or  fail.  In  our 
opinion,  the  system  of  quality  control  for  the  audit  function  of  AFAA  SAP  in  effect  for  the 
period  ended  September  30,  2013,  was  designed  in  accordance  with  quality  standards 
established  by  GAGAS.  Accordingly,  we  are  issuing  pass  opinion  on  your  SAP  audit  quality 
control  system  for  the  review  period  ended  September  30,  2013. 

Appendix  A  contains  comments,  observations  and  recommendations  where  AFAA  can  improve 
its  quality  control  system.  Appendix  B  contains  the  scope  and  methodology  of  the  review. 

We  appreciate  the  courtesies  extended  to  the  staff.  For  additional  information  on 
this  report,  please  contact  Ms.  Carolyn  R.  Davis  at  (703)  604-8877  (DSN  664-8877)  or 

Carolyn.Davis@dodig.mil. 


Randolph  K.  Stone 
Deputy  Inspector  General 
Policy  and  Oversight 
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Appendix  A 

Comments,  Observations,  and  Recommendations 

We  are  issuing  a  pass  opinion  because  we  determined  that  the  Air  Force  Audit 
Agency’s  (AFAA)  quality  control  system  is  adequately  designed  and  functioning 
as  prescribed.  The  concerns  we  identified  during  our  review  of  the  selected 
AFAA  audit  reports  were  not  cumulatively  significant  enough  to  indicate  that 
material  deficiencies  existed  in  the  AFAA  quality  control  system  for  complying  with 
generally  accepted  government  auditing  standards  (GAGAS).  We  identified  areas  of 
concern  relating  to  independence,  planning,  and  reporting.  We  judgmentally  tested 
the  reports  for  compliance  with  GAGAS  and  AFAA  audit  policies  in  nine  areas 
to  include  independence,  professional  judgment,  competence,  audit  planning, 
supervision,  evidence,  audit  documentation,  reporting  and  quality  control. 

Independence 

GAGAS  3.02  states  that  in  all  matters  relating  to  the  audit  work,  the  audit 
organization  and  the  individual  auditor  whether  government  or  public  must 
be  independent. 

AFAA  Instruction  65-101,  Financial  Management,  Installation-Level  Audit 
Procedures,  dated  November  19,  2010,  Chapter  2-Audit  Planning,  Sec.  2.4.4 
Independence  Statements  requires  that  all  audit  team  members  (auditor,  technical 
expert,  statistician,  team  chief  and  office  chief)  will  sign  the  most  current 
independence  statement  at  the  beginning  of  each  fiscal  year  (or  upon  initial  project 
assignment  for  new  employees).  The  auditor-in-charge  is  responsible  for  ensuring 
a  copy  of  each  audit  team  member’s  independence  statement  is  included  in  the 
project  file. 

We  found  that  for  two  of  the  three  audits  reviewed,  the  auditors  did  not  include 
the  independence  statement  for  the  AFAA  Representative  for  Special  Programs  in 
the  audit  project  file.  The  AFAA  Representative  for  Special  Programs  completed 
an  annual  independence  statement,  but  auditors  did  not  include  it  in  the  audit 
project  file. 
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Audit  Planning 

GAGAS  6.55  states  the  nature  and  extent  of  the  supervision  of  staff  and  the  review 
of  audit  work  may  vary  depending  on  a  number  of  factors,  such  as  the  size  of  the 
organization,  the  significance  of  the  work,  and  the  experience  of  the  staff. 

AFAA  Instruction  65-101,  Financial  Management,  Installation-Level  Audit 
Procedures,  dated  November  19,  2010,  Chapter  2-Audit  Planning,  Sec.  2.8  Audit 
Program  states  that  the  auditor  must  prepare  a  written  audit  program  before 
starting  any  in  depth  audit  work.  Team  chiefs  will  review  the  program  for  adequacy 
and  approve  the  program  before  the  auditor  starts  audit  testing. 

We  found  that  for  one  of  the  three  audits  reviewed  the  AFAA  Representative 
for  Special  Programs  did  not  properly  follow  AFAA  audit  policies  and  initial  and 
date  the  audit  program  showing  supervisory  review  and  approval  of  the  audit 
program.  While  the  AFAA  audit  program  did  not  include  an  initial  and  date  from 
the  AFAA  Representative  for  Special  Programs  showing  approval  of  the  audit 
program,  other  documentation  in  the  audit  file  included  discussions  between 
the  AFAA  Representative  for  Special  Programs  and  the  auditor  regarding  the 
audit  program. 

Reporting 

GAGAS  7.13  states  in  reporting  audit  methodology,  auditors  should  explain  how 
the  completed  audit  work  supports  the  audit  objectives  including  the  evidence 
gathering  and  analysis  techniques,  in  sufficient  detail  to  allow  knowledgeable  users 
of  their  reports  to  understand  how  the  auditors  addressed  the  audit  objectives. 
Auditors  may  include  a  description  of  the  procedures  performed  as  part  of  their 
assessment  of  the  sufficiency  and  appropriateness  of  information  used  as  audit 
evidence.  Auditors  should  identify  significant  assumptions  made  in  conducting 
the  audit;  describe  comparative  techniques  applied;  describe  the  criteria  used; 
and,  when  sampling  significantly  supports  the  auditors’  findings,  conclusions 
or  recommendations,  describe  the  sample  design  and  state  why  the  design  was 
chosen,  including  whether  the  results  can  be  projected  to  the  intended  population. 
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AFAA  Instruction  65-101,  Financial  Management,  Installation-Level  Audit 
Procedures,  dated  November  19,  2010,  Chapter  4-Draft  Report,  Sec.4.3.3.2.8.2  Audit 
Scope,  requires  the  auditors  to  include  the  following  paragraphs  in  the  audit  report: 

•  audit  coverage, 

•  sampling  methodology, 

•  data  reliability,  and 

•  auditing  standards. 

We  found  that  for  one  of  three  audit  reports  reviewed  the  auditors  did  not  properly 
report  the  sampling  methodology  used.  Specifically,  the  body  of  the  audit  report 
stated  that  sampling  was  used  to  support  audit  conclusions  and  recommendations, 
but  the  scope  and  methodology  section  of  the  report  contained  a  statement  that 
no  audit  sampling  was  used  during  the  audit.  While  the  report  contained  an 
inaccuracy  regarding  the  use  of  audit  sampling,  the  inaccuracy  did  not  have  an 
effect  on  the  overall  findings  and  conclusions  included  in  the  audit  report. 

Recommendations,  Management  Comments  and 
Our  Response 

Recommendation  A.l 

We  recommend  that  the  AFAA  Representative  for  Special  Programs: 

Remind  auditors  to  follow  AFAA  guidance  for  including  independence  statements 
for  all  individuals  who  work  on  the  audit  in  the  audit  project  file. 

Management  Comments 

AFAA  management  concurred.  The  AFAA  Representative  for  Special  Programs 
sent  a  memorandum  to  all  special  programs  auditors  on  November  5,  2014, 
reminding  the  auditors  to  comply  with  AFAA  guidance  and  to  include  independence 
statements  for  all  individuals  who  work  on  the  audit  in  the  audit  project  file. 

Our  Response 

AFAA  comments  were  responsive.  No  additional  comments  are  needed. 
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Recommendation  A. 2 

We  recommend  that  the  AFAA  Representative  for  Special  Programs: 

Follow  audit  guidance  for  review  of  audit  programs  and  document  the  review. 

Management  Comments 

AFAA  management  concurred.  The  AFAA  Representative  for  Special  Programs 
agreed  to  follow  audit  guidance  for  audit  program  reviews  and  document  the 
reviews.  The  AFAA  Representative  for  Special  Programs  sent  a  memorandum  to 
all  special  programs  auditors  on  November  5,  2014,  requiring  the  auditors  to  get 
supervisory  approval  on  the  audit  program  before  the  audit  starts  and  asked  for 
notification  from  the  auditors  if  the  program  was  either  not  signed  off  in  the  audit 
files  or  a  classified  e-mail  was  sent  indicating  approval. 

Our  Response 

AFAA  comments  were  responsive.  No  additional  comments  are  needed. 

Recommendation  A. 3 

We  recommend  that  the  AFAA  Representative  for  Special  Programs: 

Remind  auditors  to  make  sure  that  the  scope  and  methodology  section  of  the  audit 
report  corresponds  to  the  body  of  the  audit  report  to  include  the  reporting  of 
sampling  used  throughout  the  audit. 

Management  Comments 

AFAA  management  concurred.  The  AFAA  Representative  for  Special  Programs  sent 
a  memorandum  to  all  special  programs  auditors  on  November  5,  2014,  reminding 
the  auditors  to  ensure  the  scope  and  methodology  section  of  the  audit  report 
corresponds  to  the  body  of  the  audit  report  and  includes  the  sampling  method 
when  used. 

Our  Response 

AFAA  comments  were  responsive.  No  additional  comments  are  needed. 
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Appendix  B 

Scope  and  Methodology 

We  limited  our  review  to  the  adequacy  of  AFAA  SAP  audits’  compliance  with 
quality  policies,  procedures,  and  standards.  We  judgmentally  selected  three  SAP 
audits  from  a  universe  of  nineteen  SAP  audit  reports  issued  by  AFAA  SAP  auditors 
during  FY  2012  and  FY  2013.  We  tested  each  audit  for  compliance  with  the 
AFAA  system  of  quality  control.  The  Army  Audit  Agency  conducted  a  review  of 
the  AFAA  internal  quality  control  system  for  non-SAP  audits  and/or  attestation 
engagements  and  will  issue  a  separate  report.  The  Deputy  Inspector  General  for 
Policy  and  Oversight  will  issue  an  overall  opinion  report  on  the  AFAA  internal 
quality  control  system  that  will  include  the  combined  results  of  the  SAP  and 
non-SAP  audit  reviews.  In  performing  our  review,  we  considered  the  requirements 
of  quality  control  standards  contained  in  the  2011  revision  of  GAGAS  issued 
by  the  Comptroller  General  of  the  United  States.  GAGAS  3.96  (2011  revision) 
contains  a  requirement  for  an  external  peer  review  at  least  once  every  3  years. 
GAGAS  3.96  (2011  revision)  states: 

The  audit  organization  should  obtain  an  external  peer  review  at 
least  once  every  3  years  that  is  sufficient  in  scope  to  provide  a 
reasonable  basis  for  determining  whether,  for  the  period  under 
review,  the  reviewed  audit  organization’s  system  of  quality  control 
was  suitably  designed  and  whether  the  audit  organization  is 
complying  with  its  quality  control  system  in  order  to  provide  the 
audit  organization  with  reasonable  assurance  of  conforming  with 
applicable  professional  standards. 

We  conducted  this  review  in  accordance  with  standards  and  guidelines  established 
in  the  November  2012  update  to  the  Council  of  the  Inspectors  Generals  on 
Integrity  and  Efficiency  "Guide  for  Conducting  External  Peer  Reviews  of  the 
Audit  Organizations  of  the  Federal  Offices  of  Inspector  General,”  and  the  Quality 
Standards  for  Inspection  and  Evaluation,  dated  January  2011.  The  Army  Audit 
Agency  used  this  guide  in  the  review  of  non-SAP  audits  at  the  AFAA.  We  reviewed 
audit  documentation,  interviewed  AFAA  auditors,  and  reviewed  AFAA  audit  policies. 
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We  reviewed  the  DoD  OIG  Report  No.  D-2011-6-009,  "Quality  Control  Review  of 
Air  Force  Audit  Agency’s  Special  Access  Program  Audits"  dated  August  15,  2011. 

We  performed  this  review  from  July  to  September  2014  at  three  AFAA  offices. 

We  used  the  following  criteria  to  select  the  audits  under  review: 

•  began  with  the  FY  2013  audits  in  order  to  review  the  most  current  quality 
assurance  procedures  in  place, 

•  eliminated  more  than  one  audit  report  produced  from  the  same 
project,  and 

•  eliminated  audits  that  have  the  same  or  similar  titles  to  ensure  review  of 
multiple  types  of  projects. 

Our  review  would  not  necessarily  disclose  all  weaknesses  in  the  system  of  quality 
control  or  all  instances  of  noncompliance  because  we  based  our  review  on  selective 
tests.  There  are  inherent  limitations  in  considering  the  potential  effectiveness  of 
any  quality  control  system.  In  performing  most  control  procedures,  departures  can 
result  from  misunderstanding  of  instructions,  mistakes  of  judgment,  carelessness, 
or  other  human  factors.  Projecting  any  evaluation  of  a  quality  control  system 
into  the  future  is  subject  to  the  risk  that  one  or  more  procedures  may  become 
inadequate  because  conditions  may  change  or  the  degree  of  compliance  with 
procedures  may  deteriorate. 
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Management  Comments 


Director  of  Operations,  Air  Force  Audit  Agency 


DEPARTMENT  OF  THE  AIR  FORCE 

AIR  FORCE  AUDIT  AGENCY 


20  November  2014 

MEMORANDUM  FOR  DEPUTY  INSPECTOR  GENERAL,  POLICY  AND  OVERSIGHT, 
DEPARTMENT  OF  DEFENSE  INSPECTOR  GENERAL 


FROM:  HQ  AFAA/DO 

1500  West  Perimeter  Road,  Suite  4700 
Joint  Base  Andrews  MD  20762 


SUBJECT:  Management  Comments,  2014  DoD-IG  Quality  Control  Review  of  the  Air  Force 
Audit  Agency’s  Special  Access  Program  Audits 

We  reviewed  the  draft  report  of  audit  and  concur  with  the  issues  and  recommendations. 
Our  response  is  attached. 


We  express  our  thanks  to  your  staff  who  conducted  this  review  in  a  professional  and 
effective  manner.  Please  direct  questions  to  I 


Digitally  signed  by 

BROMLEY.CATHER““— — „o. 

INE.M.1 123889238  cn=BROMLEY.CATHERINE.M.1 123889238 

Date:  201 4.1 1 .20  09:50:45  -05'00' 

CATHERINE  M  BROMLEY 
Director  of  Operations 


Attachment: 

Management  Comments 
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Director  of  Operations,  Air  Force  Audit  Agency  (cont'd) 


Management  Comments  to  the  DoD-IG  Quality  Control  Review  of  the  Air  Force  Audit 
Agency’s  Special  Access  Program  Audits 

The  AFAA  Representative  for  Special  Programs  should: 

Recommendation  1.  Remind  auditors  to  follow  AFAA  guidance  for  including  independence 
statements  for  all  individuals  who  work  on  the  audit  in  the  audit  project  file. 

Recommendation  2.  Follow  audit  guidance  for  review  of  audit  programs  and  document  the 
review. 

Recommendation  3.  Remind  auditors  to  make  sure  that  the  scope  and  methodology  section  of 
the  audit  report  corresponds  to  the  body  of  the  audit  report  to  include  the  reporting  of  sampling 
used  throughout  the  audit. 

AFAA  Comments.  AFAA  management  concurs  with  the  issues  identified  and 
recommendations  1,2,  and  3,  and  provides  the  following  comments: 

Recommendation  1.  Concur.  The  AFAA  Representative  for  Special  Programs  sent  a 
letter  to  all  special  programs  auditors  reminding  them  to  follow  AFAA  guidance  and 
ensure  independence  statements  for  all  individuals  who  work  on  the  audit  are  in  the  audit 
project  file.  (Closed  -  5  November  2014) 

Recommendation  2.  Concur.  The  AFAA  Representative  for  Special  Programs  will 
follow  audit  guidance  for  audit  program  reviews  and  document  the  reviews.  He  also 
reminded  special  program  auditors  to  ensure  the  audit  project  files  contain  a  copy  of  the 
audit  program  approved  by  the  supervisor.  (Closed  -  5  November  2014). 

Recommendation  3.  Concur.  The  AFAA  Representative  for  Special  Programs  reminded 
special  program  auditors  to  ensure  the  scope  and  methodology  section  of  the  audit  report 
corresponds  to  the  body  of  the  audit  report  to  include  reporting  the  sampling  method  used. 
(Closed  -  5  November  2014) 


Attachment 


8  |  DODIG-2015-049 


Acronyms  and  Abbreviations 


Acronyms  and  Abbreviations 

AFAA  Air  Force  Audit  Agency 

GAGAS  Generally  Accepted  Government  Auditing  Standards 
SAP  Special  Access  Programs 
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Whistleblower  Protection 
U.S.  Department  of  Defense 


The  Whistleblower  Protection  Enhancement  Act  of  2012  requires 
the  Inspector  General  to  designate  a  Whistleblower  Protection 
Ombudsman  to  educate  agency  employees  about  prohibitions 
on  retaliation,  and  rights  and  remedies  against  retaliation  for 
protected  disclosures.  The  designated  ombudsman  is  the  DoD  Hotline 
Director.  For  more  information  on  your  rights  and  remedies  against 
retaliation,  visit  www.dodig.mil/programs/whistleblower. 


For  more  information  about  DoD  IG 
reports  or  activities,  please  contact  us: 

Congressional  Liaison 

congressional@dodig.mil;  703.604.8324 

Media  Contact 

public.affairs@dodig.mil;  703.604.8324 

Monthly  Update 

dodigconnect-request@listserve.com 

Reports  Mailing  List 

dodig_report@listserve.com 

Twitter 

twitter.com/DoD_IG 

DoD  Hotline 

dodig.mil/hotline 


DEPARTMENT  OF  DEFENSE  |  INSPECTOR 

4800  Mark  Center  Drive 
Alexandria,  VA  22350-1500 
www.dodig.mil 

Defense  Hotline  1.800.424.9098 


